The present invention relates to techniques for protecting computer programs, and particularly to techniques for protecting computer programs from reverse engineering.
A reverse engineer will typically attack a computer program from at least one of two ways. The reverse engineer can attempt to inspect and/or modify the data used by the computer program in order to force the program to do or to achieve the result desired by the reverse engineer. Alternatively, the reverse engineer can attempt to attack the program by attacking the control flow of the program. Computer programs use conventional control flow techniques, such as IF statements, conditional jumps etc. in order to cause the program to go or operate in a certain flow. The control flow of the computer program is typically specified by branches, and at each branch the flow can go in at least one of two directions depending upon the instructions in the computer program at a particular branch. Reverse engineers will attempt to hijack a branching control flow instruction in order to cause the program to achieve a result desired by the reverse engineer. For example, if the computer program is a program that provides digital rights management and is protecting a digital asset, such as a movie, a reverse engineer can attempt to hijack a branch control flow instruction to try to force the program to give access to the reverse engineer in order to allow the reverse engineer access to the digital asset or to allow the reverse engineer to take control of the program. For example, the reverse engineer can attempt to hijack a conditional jump that determines whether or not a signature during a handshake is valid or not or a conditional jump that determines whether a digital certificate is valid or not. Traditional methods for protecting branching control flow instructions involve verifying the instructions themselves have not been tampered with or the data in the comparison (used in the branch) have not been tampered with. One problem with these approaches to protect branches is that once a jump has occurred, a reverse engineer who has hijacked the program can undo the changes made, such as replacing the reverse engineer's corrupted data with correct data, and then the maliciously hijacked path will go undetected. U.S. Pat. No. 7,594,111 provides an example of techniques used in the past to protect computer programs from such reverse engineering attacks. Also see U.S. Pat. Nos. 8,056,138; 7,823,135; 7,620,987; and 7,841,009.